|
課程名稱 |
消息理論
Information Theory |
|
開課學期 |
112-1 |
|
授課對象 |
電機資訊學院 電信工程學研究所 |
|
授課教師 |
王奕翔 |
|
課號 |
EE5028 |
|
課程識別碼 |
921 U1190 |
|
班次 |
|
|
學分 |
3.0 |
|
全/半年 |
半年 |
|
必/選修 |
選修 |
|
上課時間 |
星期四6,7,8(13:20~16:20) |
|
上課地點 |
電二144 |
|
備註 |
總人數上限：60人
外系人數限制：20人 |
|
|
|
|
課程簡介影片 |
|
|
核心能力關聯 |
本課程尚未建立核心能力關連 |
|
課程大綱
|
|
為確保您我的權利,請尊重智慧財產權及不得非法影印
|
|
課程概述 |
Information Theory is a graduate level course designed for students who are interested in the quantitative aspects of information. What is information and how to quantify information? What is the fundamental limits in various information processing tasks, such as representing information, delivering information, and learning information? How to achieve these theoretical limits? In this course, we answer the above high-level questions with mathematical rigor and introduce the fascinating field originated from Claude E. Shannon in 1948, which is now one of the founding pillars of the information age. |
|
課程目標 |
1. Develop mathematical frameworks for quantifying the amount of information in various information processing tasks.
2. Explore the notion of various measures of information and their applications.
3. Introduce information processing algorithms that are guaranteed to achieve the fundamental limits. |
|
課程要求 |
Prerequisite: Probability, Linear Algebra.
Optional but preferred: Algorithms, Convex Analysis.
Grading: Homework (60%), Exam (30%), Participation (10%). |
|
預期每週課後學習時數 |
5.1 hours (according to the Fall 2022 class). |
|
Office Hours |
每週四 16:30~17:30
每週三 13:30~14:30 |
|
指定閱讀 |
|
|
參考書目 |
1. T. Cover and J. Thomas, Elements of Information Theory, Second Edition, Wiley-Interscience, 2006.
2. Y. Polyanskiy and Y. Wu, Information Theory: From Coding to Learning (draft), Cambridge University Press, forthcoming.
3. I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems, Second Edition, Cambridge University Press, 2011.
4. S. M. Moser, Information Theory (Lecture Notes), 6th Edition, ISI Lab, ETH Zurich, Switzerland, 2018.
5. R. Gallager, Information Theory and Reliable Communications, Wiley, 1968. |
|
評量方式
(僅供參考) |
|
No. |
項目 |
百分比 |
說明 |
|
1. |
Homework |
60% |
Six homework assignments |
2. |
Exam |
30% |
One exam |
3. |
Participation |
10% |
Details to be announced |
|
|
週次 |
日期 |
單元主題 |
|
第1週 |
9/7 |
Logistics; Course overview; Math preliminaries |
|
第2週 |
9/14 |
Typicality; Lossless source coding |
|
第3週 |
9/21 |
Shannon entropy; Sources with memory |
|
第4週 |
9/28 |
No lecture (教師節台大停課) |
|
第5週 |
10/5 |
Hypothesis testing |
|
第6週 |
10/12 |
Information divergence |
|
第7週 |
10/19 |
Mutual information; Channel capacity |
|
第8週 |
10/26 |
Noisy channel coding theorem |
|
第9週 |
11/2 |
Rate distortion theory; Lossy source coding |
|
第10週 |
11/9 |
Coding theorems for continuous sources and channels |
|
第11週 |
11/16 |
Capacity achieving codes |
|
第12週 |
11/23 |
Exam |
|
第13週 |
11/30 |
Data compression |
|
第14週 |
12/7 |
Universal source coding |
|
第15週 |
12/14 |
Estimation |
|
第16週 |
12/21 |
Finalé |
|